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College Readiness 


Students are eollege ready when they have the knowledge, skills and behaviors 
to sueeessfully eomplete a 4-year eollege eourse of study without remediation 

College readiness requires multiple eomponents and should be assessed through multiple 
measures: 

1. Academic knowledge and skills evidenced by successful completion of a rigorous 
high school core curriculum 

2. Success in college-level courses taken in high school that require in-depth subject- 
area knowledge, higher-order thinking skills, and strong study and research skills 

3. Advanced academic skills, such as reasoning, problem solving, analysis, and 
writing abilities 

4. College planning skills demonstrated by an understanding of college and career 
options and the college admissions and financing process, as well as other non- 
cognitive attributes associated with persistence, flexible, and maturity in learning 
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College Success 

• Research has clearly shown that 3 factors account for 
academic success: 

• High School grades 

• The rigor of High School courses 

• The number of courses (4 vs 3 yrs math) 

• The highest level completed (Calc vs Alg II) 

• The rigor of the courses (honors, AP vs standard) 

• Performance on Cognitive Ability Tests (Admissions Tests) 

• Studies by Cliff Adelman, EdTrust, NCES and other groups have 
been widely cited and accepted by policymakers. 

• But... no one has developed a comprehensive set of metrics that 
report on these 3 metrics for students and schools 
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College Readiness Assessments 

CB Middle School Assessment - 8'^ grade 

• Critical Reading, Math and Writing 

• Will be comparability to PSAT/NMSQT and useful for placement into 
honors and AP; informing students early on if they are on the right path 
to college readiness 

PSAT/NMSQT - 1 0'^ and grades 

• Used in conjunction with AP Potential - it identifies students who have a 
good probability of success in specific AP (and other rigorous) courses. 

SAT - 11^^ and 12^^ grades 

• Valid for adnnissions and placement decisions 

• All three tests will include a student skills insight with 

St connect to college success™ 
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SAT® Skills Insight^” 
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Choose Your Starting Point 

As with any plan, before you decide where you're going, you need to know where you are right now. So to use SAT 
Skills Insight, start with your prior scores on the PSAT/NMSQT®, the SAT, or any practice test you have taken. Then 
choose the score band for each section of the SAT that reflects your score on that section. For example, if you got a 
420 on the critical reading section of the SAT, your starting point on the Skills Insight would be the 400-490 score band 
for critical reading. (If you haven't taken the PSAT/NSMQT, the SAT, or any practice tests, you may want to start at the 
lowest score band and work your way up until you find a score band that contains skills that you find unfamiliar or 
challenging.) 
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MATHEMATICS SKILL GROUPS: 123456789 

1. Number and Operations 



Using SAT::^ Skills Insighf^" 


Academic Skills Suggestions for Improvement 


A typical student in this score band can do the 

following: 

• SKILL 1: Solve problems using ideas from basic 
set theory and basic number theory 

• SKILL 2: Recognize and apply ratio, proportion, 
or percent in solving problems 

• SKILL 3: Use properties of real number 
operations, ordering, and the zero-product 
property 

• SKILL 4: Solve problems involving counting 
techniques 


To prepare for the next score band, try the 
following: 

^►Determine values or properties of numbers in a 
sequence when given a description of the 
sequence 

^►create and use ratios, fractions, or percents in 
solving problems 

^►Solve more-complex counting problems (e.g., 
permutations, combinations, and 
inclusion/exclusion) 


Skill Examples 


The example questions below demonstrate the Academic Skills found in this score band. Without looking at the 
answers, try out the questions to see how comfortable you feel with the skills they test. 



SKILL 1— EXAMPLE 

Solve problems using ideas from basic set theory and basic number theory 


X = set of positive integer factors of 6. 

Y = set of positive integer factors of 9. 

Z = set of positive integer factors of 15. 

X, Y, and Z represent three sets of numbers, as defined above. 
What is the set of numbers that belong to set Z but to neither 
set X nor set Y ? 


3} 

8 <.1, 5} 
c {5, 15> 

8 {2, 3, 5} 

E {1, 3, 5, 15} 


IM 


Answer: C 






Tests and Grades 

• The College Board has SAT, PSAT and 
soon Middle School Test Scores 

• We are developing Benchmarks of College 
Success tied to FGPA and probability of 
graduation from 4-yr colleges. 

• We also have self reported GPA and 
schools, districts and states have actual 
grades 
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And the College Board also have 
courses students have taken 

• The number of courses in a content 
area (4 vs 3 yrs math) 

• The highest level completed (Calc vs 
Alg II) 

• The rigor of the courses (honors, AP vs 
standard) 

• The courses taken by grade (Alg I in 9^^ 
grade... Pre calc in 12^^ grade) 

Si 
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Academic Rigor - The purpose! 

• Compute a measure of academic rigor that predicts college 
success using the number and type of courses taken in 
academic subjects and the rigor of those courses. 

• Validate this metric with data from the SAT validity study (110 
colleges) and National Student Clearinghouse (92% of 
colleges) 

• Demonstrate the incremental validity of this new metric of 
academic rigor when added to SAT scores and HSGPA. 

• Establish a College Readiness metric (SAT+HSGPA+Academic 
Rigor). Provide scores to students, schools and states and 
monitor trends over time 

• Beginning in Late 2009 we can provide school, state and 
national data on College Readiness and supplement CB Srs 
report devoted only to national SAT scores. 

• We believe that multiple measures are better than a single test 
score in discussion college readiness! 
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Academic Rigor Prototype @ your schooi 


Scale 

%at 

school 

%in 

state 

%in 

US 


Completed 
College 
Core PLUS 

Completed 
College Core 

AP/DE 

Level 

Score 

Honors Level 
Score 

8 

6 

5 

7 


Y 


3-4 

3-4 

7 

9 

8 

9 


Y 


2-3 

2-3 

6 

25 

26 

22 


Y 


0-2 

0-2 

5 

22 

20 

18 



Y 

3-4 

3-4 

4 

12 

12 

7 



Y 

1-3 

2-3 

3 

16 

9 

5 



Y 

0-2 

0-2 

2 

10 

13 

12 




NA 

2-4 

1 

0 

7 

8 




NA 

0-2 

Mean 

4.6 

4.3 

4.4 







Tt7 


Computing the Academic Rigor index 

Similar to setting SAT benchmarks we will: 

1 .Conduct regressions to identify the best 
combination of predictors for different scaling 
options (FGPA, persistence) 

2. Compute the % of students meeting different 
benchmarks (x ethnicity, SES, language) to 
estimate impact 

3. Use a policy capturing approach (or standard 
setting) with external educational experts to 
determine final levels. 
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Academic Rigor - Prototype Scoring 





Core 

Plus 

Core 

ELA 

4+ 

4+ 

Science 

3+ 

(Physics, 
Chem, Bio, 
Eq.) 

3-h (2 of 
Physics, 
Chem, Bio, 
Eq.) 

Math 

4+ (Pre- 
Calc) 

4+ (Alg II) 

Social 

Science/ 

History 

4+ 

3+ 

For. 

Lang. 

3+ (same 
lang) 

2 + (same 
lang) 




AP/DE 

score 

4 

3 

2 

1 

Sci-Math 

2+ 

1 + 



Other 

2+ 




Total AP 

4+ 

3+ 

2+ 

1-2 

Total 

AP+DE 

4+ 

3+ 

3+ 

2+ 


Honors 

(includes 

AP-DE) 

4 

3 

2 

1 

Sci-Math 

4+ 

3+ 

1 + 


Other 

4+ 




_ Total AP, 
DE, Honors 

8+ 

6+ 

4+ 

2+ 


12 


Academic Success is NOT coiiege 
success 

• Need other measures that extend beyond 
grades, courses and achievement 

• Study Skills, Engagement, Self efficacy, 
meta-cognition.... 
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Predictors of College Success 


College Skills 

Content Knowledge 
Achievement 

Non-Cognitive 

Personal Qualities/ 
Experiences/ 
Characteristics 

School Performance/ 
Context 

Guidance 

Verbal Reasoning 

Math 

Motivation 

Letters 

Grades 

Career Interests 

Math Reasoning 

Language Arts 

Follow-through 

Essay 

GPA 

Study Skills 

Writing 

Science 

Communication 

Community Service 

Weighted GPA 

Interest in Major 

Metacognition 

Social Studies/ Humanities 

Conscientiousness 

Extra-curricular 

Rank 

Self Efficacy 

Creativity 

Foreign Language 

Leadership 

Work Experience 

Courses Completed 

Aspirations/ 

Practical Knowledge 
Spatial Relations 
Intellectual Curiosity 

Language Proficiency 

Other Personality 

Literacy in Second Lang 

Teacher Ratings 

Gender 

Ethnicity 

Residence 

Age 

Family Education/ Income 
Ability to Pay 
Ability to Benefit 

Academic Rigor 
AP/Honors Courses 
School Size 
School Quality 

Realistic Self-concept 


College Board 


Schools Have 


Not Developed 




CollegeBoard 


connect to college success™ 
www.collegeboard.com 




Non Cognitive Measures in Coiiege 
Admissions: Background 

• Choosing students: Higher education admissions tools for the 21 century 
(Camara & Kimmel, 2005) 

• Purpose: 

• Identify additional predictors of college success 

• Expand the definition of what constitutes successful performance in 
college beyond freshman GPA 

• Collaboration with Research Team at Michigan State University (Schmitt & 
Oswald) 

• Review university mission statements and 
department objectives 

• Interview with university staff responsible 
for student life at Michigan State University 

• Review of the education literature on student outcomes 

• Our systematic search resulted in 1 2 dimensions of student performance. . . 
Si 
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12 Dimensions of Student Performance 

Broadening the Performance Domain in the Prediction of Academic 
Success (Schmitt, Oswald, & Gillespie, 2004) 

1 . Knowledge, learning, mastery of general principles 

2. Continuous learning, intellectual interest and curiosity 

3. Artistic and cultural appreciation 

4. Multicultural appreciation 

5. Leadership 

6. Interpersonal skills 

7. Social responsibility, citizenship and involvement 

8. Physical and psychological health 

9. Career orientation 

10. Adaptability and life skills 

1 1 . Perseverance 

12. Ethics and integrity 

Si 
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Two “Noncognitive” Measures 


• Situational judgment inventory 

• A situation is presented along with several alternative 
courses of action. 

• The respondent is asked to indicate what she/he 
would be most likely and least likely to do. 


• Biodata 

• Short, multiple choice reports of past 
experience/background and interests/preferences. 
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Sample SJI Item for Leadership 

You are assigned to a group to work on a particular project. When 
you sit down together as a group, no one says anything. 


a)-1 Look at them until someone eventually says something 


b)Start the conversation yourself by introducing yourself 


c)+1 Get to know everyone first and see what they are thinking about the 
project to make sure the project’s goals are clear to everyone 


d)Try to start working on the project by asking everyone’s opinion about 
the nature of the project 


e)You would take the leadership role by assigning people to do things or 
ask questions to get things rolling 
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Sample Biodata Items for Leadership 

1. The number of high school clubs and organized activities (such as 
band, sports, newspapers, etc.) in which I took a leadership role 
was: 

a) 4 or more 

b) 3 

c) 2 

d) 1 

e) I did not take a leadership role 


2. How often do you talk your friends into doing what you want to do 
during the evening? 

a) most of the time 

b) sometimes (about half the time) 

c) occasionally (about as often as others in my group 

d) seldom or infrequently 

e) never 
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study 1 : 

Psychometric adequacy & scale refinement 


• 644 MSU freshmen completed one of the two parallel forms of the 
biodata and SJI instruments at the beginning of the academic year. 

• Identical empirical-keying procedures were conducted on both 
instruments at the item level (double-cross validated using randomly 
split samples). 

• Results indicated significant incremental validity for some of the scales 
above and beyond the validity of SAT/ACT scores and existing 
measures of personality in predicting college GPA. 

• The biodata and SJI demonstrated the greatest incremental validity 
when absenteeism, students’ self ratings, and peer-ratings of 
performance were examined ( .19, .22, and .14, respectively). 
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Study 1 : Standardized Differences 

Compared with White group... 


Non-cognitive Dimension 

Black 

Hispanic 

Asian 

Knowledge 

-0.08 

-0.20 

-0.25 

Learning 

0.01 

0 .63* 

-0.19 

Artistic 

-0.19 

0 .73* 

0.15 

Multicultural 

-0.11 

0 .63* 

0.02 

Leadership 

-0.18 

0.08 

-0.30 

Interpersonal 

-0.18 

0.33 

-0.38* 

SJI composite 

-0.05 

-0.14 

-0.21 

Citizenship 

0.05 

0.23 

-0.14 

Health 

-0.31* 

0.06 

-0.67* 

Career 

0 .34* 

0 .56* 

0.14 

Adaptability 

0.03 

0.09 

-0.41* 

Perseverance 

0.13 

0 .55* 

-0.18 

Ethics 

0.17 

-0.06 

-0.13 


• Positive values indicate that minorities perform better than White students. 

• The lvalues for biodata and SJI measures across ethnic and gender subgroups were consistently 
smaller than those found on cognitive predictors. 

« * p <.05 

Si 
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Follow-up to Study 1 : Time to Graduate 

• Obtained graduation records for the initial 
sample 

• Specifically, copies of the graduation records 
released by the school from spring 2004 through 
the summer of 2006 (4 to 6 years from 
matriculation) were obtained 

• Examined the incremental validity of biodata 
and SJI over standardized test scores in the 
prediction of time to graduate 
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Predicting Time to Graduate 

Variable Standardized Regression Weight/ R/ 


SAT/ACT scores 

.048 

R 2 

.001 

Knowledge 

-.025 

Continuous Learning 

-.128 

Artistic Appreciation 

-.148 

Multicultural 

-.018 

Appreciation 


Leadership 

.167 

Interpersonal Skills 

.053 

Citizenship 

-.045 

Health 

-.053 

Career Orientation 

-.085 

Adaptability 

.000 

Perseverance 

.150 

Ethics 

.087 

SJI 

.050 

R 2 

.090 


Note 


• Numbers in the table 
are standardized 
regression coefficients. 

• * Indicates statistical 
significance p < .05. 

• N = 424. 

• Biodata and SJ I add 
significant incremental 
validity to the 
prediction of time to 
graduate above 
standardized test 
scores. 
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study 2: Refine Scales & Replicate Study 1 
Participating Institutions & Sample Size 


HBCU N 

Winston-Salem (public) 229 

Spelman College (private) 254 

Big Ten (public) N 

University of Iowa 335 

Michigan State University 546 

Ohio State University 304 

University of Michigan 297 

Indiana University 170 

Other institutions N 

University of Chicago (private) 1 68 

Cal State - Fullerton (public) 223 

Virginia Tech (public) 237 
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Predicting FYGPA: Totai Sampie across 
10 Institutions (N= 2443) 


Variable 

Mean 

SD 

Validity 

Regression Wt. 

HS-GPA 

3.51 

.42 

.61 

.70* 

SAT/ACT 

.61 

.91 

.64 

.38* 



R (Adjusted R) 

.70 (.70) 

Biodata 

Knowledge 

3.15 

.48 

.29 

.08* 

Learning 

3.08 

.61 

.13 

-.00 

Artistic 

2.91 

.83 

.22 

.02 

Diversity 

2.98 

.66 

.13 

.03 

Leadership 

3.07 

.81 

.14 

-.02 

Responsibility 

3.35 

.76 

.19 

.02 

Health 

3.26 

.51 

.16 

.10* 

Citizenship 

3.31 

.65 

-.17 

-.15* 

Adaptability 

3.38 

.44 

.11 

-.02 

Perseverance 

3.73 

.49 

.09 

.08* 

Ethics 

3.86 

.55 

.18 

-.01 

SJI Composite 

.67 

.33 

.27 

.23 

Constant 




-.28* 



R (Adjusted R) 

.72 (.72) 


Change in R .02* 


^CoUegeBocuxl 
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Predicting Ciass Absenteeism: Totai Sampie across 10 

institutions (A/= 899) 


Variable 

Mean 

SD 

Validity 

Regression Wt. 

HS-GPA 

3.64 

.35 

-.04 

.34* 

SAT/ACT 

.98 

.81 

.17* 

.36* 



R (Adjusted R) 

.22* (.22) 


Biodata 


Knowledge 

3.23 

.46 

-.15* 

-.14 

Learning 

3.16 

.61 

-.06* 

-.04 

Artistic 

3.09 

.85 

-.03 

-.03 

Diversity 

3.05 

.69 

-.03 

.03 

Leadership 

3.11 

.80 

-.05 

.05 

Responsibility 

3.42 

.74 

-.08* 

-.02 

Health 

3.27 

.48 

-.18* 

-.34* 

Citizenship 

3.22 

.69 

-.06* 

.11* 

Adaptability 

3.42 

.44 

-.09* 

.08 

Perseverance 

3.74 

.46 

-.17* 

-.14 

Ethics 

3.92 

.51 

-.19* 

-.18* 

SJI Composite 

.73 

.29 

-.17* 

.41* 

Constant 




5.17* 



R (Adjusted R) 

.36* (.34) 


Change ini? .14* 


^CoUegeBocuxl 
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Representative Subgroup Differences in 

Standardized Units 


Compared with White group 



SAT/ACT 

HS-GPA 

SJI 

Persevere 

Career 

Learn 

Responsible 

Hispanic 

-.83 

-.61 

-.18 

-.01 

OO 

o 

-.05 

o 

o 

Asian 

.38 

.12 

-.03 

-.12 

-.03 

-.18 

.07 

African- 

American 

-1.15 

-.81 

-.23 

.20 

.52 

-.11 

-.14 

Compared i 

with Male group 
SAT/ACT HS-GPA 

SJI 

Persevere 

Career 

Learn 

Responsible 

Females 

-.42 

.02 

.36 

.22 

.17 

-.20 

.26 


< .20 = small effect 
.20-. 50 = moderate effect 
> .50 = large effect 
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Percent of Students Selected: 

Two Composites and Three Selection Strategies 



Top 85% 

Top 50% 

Top 15% 

Group 

AB 

AB+ 

AB AB+ 

AB AB+ 

Hispanic 

4.4 ^ 

4.6 

4.1 ^ 4.9 

3.9 ^ 5.5 



(+.2) 

(+.8) 

(+1.6) 

Asian 

7.6 

7.7 

9.9 ^ 9.5 

17.5^ 12.9 



(+.1) 

(-.4) 

(-4.6) 

African-American 

17.9 -> 

19.8 

9.6 -> 13.6 

1.3 ^ 7.2 



(+1.9) 

(+4.0) 

(+5.9) 

White 

70.2 -> 

67.9 

76.4^ 71.9 

77.2^ 74.4 



(-2.3) 

(-4.5) 

(-2.8) 


AB = equally weighted composite of HSGPA and SAT/ACT. 

AB+ = equally weighted composite of HSGPA, SAT/ACT, Biodata, and SJI. 
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Follow-up to Study 2: 
Cumulative GPA and Graduation 

• In the process of obtaining GPA data for all 
four years and whether or not the students 
graduated in the usual four year time frame. 

• 8 of the 1 0 institutions have/will provide data 

• After cleaning and preparing the data, 
results should be available this Fall/Winter 
( 2008 ) 

• Stay Tuned! 
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study 3: Participating Institutions (k=15): 
Appiicants 


• University of Washington 

Meredith College 

• Michigan State University 

University of Southern California 

• Lafayette College 

Furman University 

• Earlham College 

University of North Carolina, Chapel Hill 

• Ohio State University 

Kenyon College 

• Purdue University 

Gonzaga University* 

• Spelman College 

• Johnson and Wales University 

University of Puget Sound* 


* Sample size to small to run within-institution analyses 
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study 3: Data Collection 

September 2007 - May 2008 

• Applicants completed 2 non-cognitive measures 


N 


(SJI) 


= 4,164 


* ^ (biodata dimensions) fl"Om 5,744 tO 7,645 

Fall 2008 

• Determine which applicants are enrolled in participating 
institutions 


Spring 2009 


Collect outcome data 
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Study 3: Demographic Information 


• Overall, 7,884 students participated in our study. 60.3% of 
participants were female. The average age was 1 7.39 
years old. 


Racial/ethnic breakdown below: 


Native Multi-racial 
American. 


Asian 

18 % 


Hispanic 

6% 


Black 

6% 
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Other 



Undisclosed 

1% 


White 

62% 
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study 3: Results - Mean differences 


Dimension 

Average score for 13 
universities 2007-2008 

Average score all 10 
universities 2004 

Knowledge (.73) 

3.51 (.48) 

3.15 (.47) 

Continuous Learning (.83) 

3.53 (.61) 

3.09 (.61) 

Artistic Appreciation (.86) 

3.31 (.80) 

2.91 (.82) 

Multicultural Appreciation (.83) 

3.40 (.67) 

2.98 (.66) 

Leadership (.88) 

3.39 (.79) 

3.07 (.81) 

Social Responsibility (.80) 

3.73 (.72) 

3.32 (.76) 

Health (.74) 

3.32 (.54) 

3.25 (.51) 

Career Orientation (.78) 

3.49 (.62) 

3.32 (.65) 

Adaptability (.72) 

3.51 (.45) 

3.38(.45) 

Perseverance (.78) 

3.94 (.47) 

3.73 (.49) 

Ethics (.66) 

4.11 (.43) 

3.86 (.54) 

Jobs Scale (.91) 

2.63(1.07) 

2.76 (.84) 

Awards Scale (.77) 

2.39 (.75) 

2.42 (.70) 

LEAD (.49) 

,41(.15) 

.33 (.17) 


Note. Reliability (coefficient alpha) for each scale is presented in parentheses next to the scale labels. 
Standard deviations are in parentheses next to the means. 
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Limitations & Future Research 


• Public relations and acceptance of these measures by 
consumers (i.e., admissions officers, parents, students). 
Need to collect reactions to new admissions measures 
along a variety of dimensions (e.g., fairness, face 
validity). 

• Fakability in high-stakes situation especially relevant for 
biodata, less so for SJI. However, note that essays can 
be coached and edited, and self-reported activities can 
also be inflated. 

• Reluctance to experiment in Higher Ed admissions 
context despite efficacy and value. Need a consortium of 
institutions to move from a research effort to application. 
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Next Steps 

• Replicate previous analyses with data from 
Study 3 

• Benefits of Study 3: 

• The sample size will be much larger, therefore, 
increasing the power to detect significant effects 

• Additionally, sampling error will be reduced 

• Finally, the data represents a multi-institutional sample 
increasing the generalizability of results 

• For more information, go to 

WWW. iopsych. msu. edu/cbstudy/ 

& 
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